Tennis ball tracking for automatic annotation of broadcast tennis video

This thesis describes several algorithms for tracking tennis balls in broadcast tennis videos. The algorithms are used to provide a ball tracking module for an automatic tennis video annotation system. As an overall strategy, we have decided to adopt the Track-After-Detection (TAD) approach to the tracking problem. A TAD approach decomposes a tracking problem into two sub-problems: object candidate detection and object candidate tracking. In the second sub-problem, resolving data association ambiguity is the key. In this thesis, we propose a novel tennis ball candidate detection algorithm, and three novel data association algorithms. The ball candidate detection algorithm extracts foreground moving blobs using temporal differ-encing, and classify the blobs into ball candidates and non-candidates. A novel gradient-based feature for measuring the departure of the shape of a blob from an ellipse is proposed. The proposed data association algorithms take as input the positions of the ball candidates, and try to resolve the object-candidate association ambiguity. The first data association algorithm is based on a particle filter with an improved sampling scheme. Smoothing and an explicit data association process are also used to increase the accuracy of the tracking results. The second data association algorithm is based on the Viterbi algorithm. It seeks the sequence of object-candidate associations with maximum a posteriori probability, using a modified version of the Viterbi algorithm. The third data association algorithm is a multi-layered one with graph-theoretic formulation. At the bottom layer, " tracklets " are " grown " from ball candidates with an efficient variant of the RANdom SAmple Consensus (RANSAC) algorithm. The association problem is then formulated as an all-pairs shortest path (APSP) problem in a graph with tracklets as its nodes, and is solved with a novel fast APSP algorithm. The proposed data association algorithms have been tested using tennis sequences from the Australian Open tournaments. Their performances are compared both mutually and to existing object tracking algorithms. Experiments show that the particle filter based algorithm gives similar performance to the state-of-the-art tennis ball tracking method in the literature: the Robust Data Association (RDA), while the other two proposed algorithms both outperform RDA. Acknowledgements I would like to thank my supervisors, Dr. William Christmas and Prof. Josef Kittler, for their guidance and support throughout the course of this work. I would also like to thank everyone who has offered his or her help, encouragement and friendship. In particular, thanks are due to Yonggang …